High-performance, embedded Key-Value engine built with Rust 🦀
Implementing LSM-Tree architecture with a focus on SOLID principles, observability, and performance.
ApexStore is a modern, Rust-based storage engine designed for write-heavy workloads. It combines the durability of write-ahead logging (WAL) with the efficiency of Log-Structured Merge-Tree (LSM-Tree) architecture.
Built from the ground up using SOLID principles, it provides a production-grade storage solution that is easy to reason about, test, and maintain, while delivering the performance expected from a systems-level language.
While industry giants like RocksDB or LevelDB focus on extreme complexity, ApexStore offers:
- Educational Clarity: A clean, modular implementation of LSM-Tree that serves as a blueprint for high-performance systems.
- Strict SOLID Compliance: Leveraging Rust's ownership model to enforce clear boundaries between MemTable, WAL, and SSTable layers.
- Observability First: Built-in real-time metrics for memory, disk usage, and WAL health.
- Modern Defaults: Native LZ4 compression, Bloom Filters, and 35+ tunable parameters via environment variables.
Measured on AMD Ryzen 9 5900X, NVMe SSD (v1.4.0)
| Operation | Throughput | Visual |
|---|---|---|
| In-Memory Writes | ~500k ops/s | ████████████████ 100% |
| Writes (with WAL) | ~100k ops/s | ███ 20% |
| Batch Writes | ~1M ops/s | ██████████████████████████████ 200% |
| MemTable Hits | ~1.2M ops/s | █████████████████████████████████ 240% |
| SSTable Reads | ~50k ops/s | █ 10% |
Note: The performance difference between In-Memory and WAL writes highlights the fsync overhead, which can be optimized via
WAL_SYNC_MODE.
- MemTable: In-memory BTreeMap with configurable size limits.
- Write-Ahead Log (WAL): ACID-compliant durability with configurable sync modes.
- SSTable V2: Block-based storage with Sparse Indexing and LZ4 Compression.
- Bloom Filters: Drastically reduces unnecessary disk I/O for read operations.
- Crash Recovery: Automatic WAL replay on startup to ensure zero data loss.
- Interactive CLI: REPL interface for development and debugging.
- REST API: Full HTTP API with JSON payloads for microservices.
- Batch Operations: Efficient bulk inserts and updates.
- Search Capabilities: Prefix and substring search (Optimized iterators coming in v2.0).
The engine follows a modular architecture where each component has a single responsibility:
graph TB
subgraph "Interface Layer"
CLI[CLI / REPL]
API[REST API Server]
end
subgraph "Core Domain"
Engine[LSM Engine]
MemTable[MemTable<br/>BTreeMap]
LogRecord[LogRecord<br/>Data Model]
end
subgraph "Storage Layer"
WAL[Write-Ahead Log<br/>Durability]
SST[SSTable Manager<br/>V2 Format]
Builder[SSTable Builder<br/>Compression]
end
subgraph "Infrastructure"
Codec[Serialization<br/>Bincode]
Error[Error Handling]
Config[Configuration<br/>Environment]
end
CLI --> Engine
API --> Engine
Engine --> WAL
Engine --> MemTable
MemTable -->|Flush| Builder
Builder --> SST
Engine -->|Read| MemTable
Engine -->|Read| SST
WAL -.->|Recovery| MemTable
Engine --> Config
SST --> Codec
Builder --> Codec
WAL --> Codec
style Engine fill:#f9a,stroke:#333,stroke-width:3px
style WAL fill:#9cf,stroke:#333,stroke-width:2px
style SST fill:#9cf,stroke:#333,stroke-width:2px
- Rust 1.70+: Install via rustup.rs
# Clone and enter
git clone https://github.com/ElioNeto/ApexStore.git && cd ApexStore
# Build and Start REPL
cargo run --release
# Available commands:
# > put user:1 "John Doe"
# > get user:1
# > statsRun ApexStore as a standalone API server:
# Start with Docker Compose
docker-compose up -d
# Manual run with custom config
docker run -d \
--name apexstore-server \
-p 8080:8080 \
-e MEMTABLE_MAX_SIZE=33554432 \
-v apexstore-data:/data \
elioneto/apexstore:latest| Method | Endpoint | Description |
|---|---|---|
POST |
/keys |
Insert/Update: {"key": "k1", "value": "v1"} |
GET |
/keys/{key} |
Retrieve value |
GET |
/stats/all |
Full telemetry (Memory, Disk, WAL) |
ApexStore/
├── src/
│ ├── core/ # LSM Engine, MemTable, Domain logic
│ ├── storage/ # WAL, SSTable V2, Block Builder
│ ├── infra/ # Codec, Error Handling, Config
│ ├── api/ # Actix-Web Server & Handlers
│ └── cli/ # REPL Implementation
├── docs/ # Detailed documentation & Architecture
├── tests/ # Integration test suite
└── Dockerfile # Multi-stage build
cargo test # Run all tests
cargo clippy -- -D warnings # Linting
cargo fmt # FormattingApexStore uses trunk-based development with automated releases:
graph LR
A[Feature Branch] -->|Open PR| B[CI Validation]
B -->|✅ Pass| C[Merge to main]
C --> D[Auto Release]
D --> E[v2.1.X]
- Create feature branch from
main - Open PR → CI runs
cargo fmt,clippy,test,build - Merge PR → Auto-increments version in
Cargo.toml, creates tag & GitHub release
📖 Read: MIGRATION_GUIDE.md for team workflow
📂 Details: .github/workflows/README.md
- SSTable V2 with compression & Bloom Filters
- REST API & Feature Flags
- Global Block Cache
- Trunk-based CI/CD with auto-release
- v2.2: Storage iterators for range queries
- v2.3: Concurrent read optimization
- v3.0: Leveled/Tiered Compaction Strategies
Contributions are what make the open-source community an amazing place! Please check our Contributing Guidelines.
- Fork the Project
- Create your Feature Branch (
git checkout -b feat/amazing-feature) - Commit your Changes (
git commit -m 'feat: add amazing feature') - Push to the Branch (
git push origin feat/amazing-feature) - Open a Pull Request to
main - CI will auto-release on merge 🚀
Distributed under the MIT License. See LICENSE for more information.
Elio Neto - GitHub - netoo.elio@hotmail.com
Demo: lsm-admin-dev.up.railway.app
Built with 🦀 Rust and ❤️ for high-performance storage systems